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ABSTRACT 

This is the second of four working papers on 
dissemination policies, commissioned in 1993 by the Office of 
Educational Research and Improvement ♦ It is intended to stimulate 
discussion and the exchange of insights on the government's role in 
empowering educators to contribute to the evaluation of innovations 
they use to improve education. Traditionally, information has been 
shared in education through feedback to the teacher, and the useful 
information teachers can provide about the implementation and impact 
of innovations ha? often been ignored* This report describes and 
analyzes key types of strategies as well as technical and 
psychological issues in establishing strategies to develop a 
systematic national education dissemination system. In addition, 
several consumer evaluation options are presented* The growth of 
electronic networks has the potential to make a model of two-way 
communication and information-sharing possible. While the technical 
challenges of setting up such a system are not great, the societal 
and psychological challenges or regarding the teacher, student, and 
policy maker as a customer are large* A possible model of an 
information feedback system and the types of information it could 
collect are presented. An appendix contains some key contacts. 
(Contains 36 ref '^rences . ) (SLD) 
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Note from The Office of Educational Research and Improvement, OERI 



This working paper, "Whose Knowledge Is It?: Involving Teachers In the 
Generating and Using of Information on Educational Innovations" was prepared by 
Dr. Patricia B. Campbell for Dr. Sharon P. Robinson, Assistant Secretary of the Office 
of Educational Research and Improvement, OERI at the suggestion of Dr. Laurence 
Peters. This paper was prepared under Education Department Purchase Order 43- 
3JAJ-3-01053 for $2,450. Individuals undertaking such projects are encouraged to 
express freely their professional judgment. This report, therefore, does not necessarily 
represent positions br policies of the U.S. Department of Education, and no official 
endorsement should be inferred. 

This is the second of four working papers on dissemination policies commissioned in 
September 1 993 by OERI. The other three topics are (1 ) "A Matter of Consensus" on 
using R&D based evidence to develop consensus on establishing R&D agendas and 
deciding what is worth disseminating (Dr. Lois-ellin Datta); (2) "Readiness for Change, 
Educational Innovations, and Education Reform" (Dr. Thomas E. Backer); and (3) the 
role of technical assistance centers in promoting education R&D solutions (Drs. Brenda 
Turnbull and Mary Leighton). 

This working paper is intended to stimulate discussion and the exchange of insights on 
the government's role in empowering educators to contribute to the evaluation of 
innovations that they use to improve education. We encourage you to request copies 
or share reactions and additional information with OERI project officer. 

Dr. Susan Klein, Programs for the Improvement of Practice, OERI 

555 New Jersey Ave., N.W., Washington, DC 20208-5645, Tel. 202-219-2038, 

e-mail: skle(in@inet.ed.gov 

or with the author, 

Dr. Patricia B. Campbell, Campbeii-Kibler Associates 

80 Lakeside Drive, Groton Ridge Heights, Groton, MA 01450, Tel. (508) 448-5402 
e-mail: 73307.1330@compuserve.com 
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Foreword: "Whose Knowledge is It?" Putting Teaciier Feedbacit on 

Education Innovations in Context 

by 

Susan S. Klein, Ed.D, U.S. Department of Education 

This visionary report by Patricia Campbell was commissioned by the Office of 
Educational Research and Improvement (OERI) in the U.S. Department of Education 
(ED) to guide the implementation of its new legislation, the "Educational Research, 
Development and Dissemination Improvement Act of 1994" in Title IX of Goals 2000. 
Thus, opportunities for improving the federal role in education research and 
development (R&D) provide the primary context for this examination of "Whose 
Knowledge Is It?: Involving Teachers in the Generating and Using of Information on 
Educational Innovations". 

One of the weak links in the education R&D system has been the limited involvement of 
teachers and other consumers in research and evaluation to improve what they and 
others are doing on an ongoing basis. OF.RI has started to provide leadership in this 
area by sponsoring a national "Invitational Conference on Teacher-led Inquiry" and by 
funding research on involving teachers in their ovm evaluation as well as the evaluation 
of their students. This report is an attempt to examine a related, but somewhat different 
way for ED to help our nation expand its use of teachers' expertise. It is designed to 
examine how teachers' experiences with specific innovations can be shared 
systematically with other teachers who are in a position to decide if and how the^ will 
use the innovation. This strategy involves both top-down and bottom-up leadership. 
From the top-down, the federal government can establish or foster the development of 
systems that will compile, synthesize and interpret teacher evaluations of specific 
products and programs to provide consumer information to other teachers on what 
works best. From the bofiom-up, the teachers can be empowered by feeding back their 
evaluations based on their own experiences with the innovations to educators outside 
of their school and directly contributing their wisdom to educational improvement 
across the nation. 

Three forces are currently operating to make Dr. Campbell's approach to this challenge 
feasible. They include: 

□ A new legislative mandate for the U.S. Department of Education 

Increasing the nation's knowledge of what's promising and exemplary in the 
treasure chest of R&D based resources is a critical element In the 1994 
reauthorization of OERI. This responsibility is given to OERI's new Office of Reform 
Assistance and Dissemination as well as to each of its five new Institutes. 
Congress wants OERI to provide leadership in more systematically identifying and 
sharing the best from the nation's treasure chest of educational products, programs 

V 
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and practices. Members of Congress such as Representative Major Owens, chair of 
the subcommittee which had the most responsibility for the reauthorization of OERI, 
have been concerned about the limited systematic collection and use of feedback 
from teachers on what educational innovations work well or poorly for them and 
their students. To remedy this omission, OERI is now authorized to: 

(A) create a national system of dissemination, development, and educational 
improvement in order to create, adapt, identify, validate, and disseminate to 
educators, parents, and policymakers those educatioml programs that have 
potential or have been shown to improve educational opportunities for all 
students: and 

(B) empower and increase the capacity of teachers to participate in the research 
and development process. 

(U.S. Congress, 1994, Goals 2000: Educate America Act, Title IX, Part D, National 
Education Dissemination System, Sec. 941 (2) Purpose, p. 125). 

This is to be accomplished through electronic networking and new technologies and 
through the auspices of the Educational Resources information Center (ERIC) 
Clearinghouses, the regional educational laboratories, the Teacher Research 
Dissemination Demonstration Program, the Goals 2000 Community Partnership 
Program, the existing National Diffusion Network, the Institutes and other 
appropriate programs or activities. 



□ Increased Availability of Computers and Electronic Mail 

Educators are relying on computers to help them keep track of all kinds of 
descriptive and evaluative information and increasing numbers of schools are using 
computer modems to connect with others outside their buildings. Thus, it is much 
easier than before to collect a continuous flow of evaluation information from 
teachers across the nation and to share the cumulative results with other educators. 
Dr. Campbell has emphasized these advances in technology in her examination of 
the issues and possibilities for federal leadership in using teacher feedback. 

□ The ability to build on substantial national experience in educational 
evaluation to develop new approaches to understand the comparative 
strengths and weaknesses of numerous R&D-based resources 

Although most previous evaluations have been more likely to focus on the 
identification of weaknesses and needs rather than comparative strengths of viable 
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solutions, the nation can capitalize on the increased numbers of trained evaluators 
as well the general research and evaluation skills of teachers and administrators. In 
addition to evaluating their students, many teachers have participated in pilot 
evaluations of instructional materials and in federally funded accountability 
evaluations to show that federal programs are doing what they said they would. 
And as we mentioned earlier, there is also renewed interest in having teachers 
conduct classroom research. Sharing specific evaluative feedback on programs or 
products that they and many other teachers use Is one way to build on these 
capacities for inquiry and improvement. Professional evaluators and teachers with 
evaluation experience and interests can help make it feasible to implement a 
system like what Dr. Campbell has started to design. 

However, new evaluation systems, incentives at>d roles for teachers and others also 
will be needed. With some notable exceptions such as the Department of 
Education's Program Effectiveness Panel (PEP) which selects programs for ED's 
National Diffusion Network (NDN) and the work of the Educational Products 
Information Exchange (EPIE), evaluations have rarely been used on an ongoing 
basis or to help consumers make informed decisions about the strengths and 
weaknesses of their product or program options. The role of the federal 
government in evaluating what they and others fund to guide future federal 
investments in dissemination is discussed in a recent special feature oi Evaluation 
and Program Planning on "Sharing the best: Finding better ways for the federal 
government to use evaluation to guide the dissemination of promising and 
exemplary education solutions" (Klein, 1994). One of the recommendations from 
these analyses is for the Department of Education to develop ways to empower 
teachers and other educators to contribute to ongoing evalur.^Jons of promising and 
exemplary products, programs and practices for both revision and dissemination 
purposes. Dr. Campbell's report addresses this recommendation. 

In this report Dr. Campbell has: 

1 . Helped us understand aspects of the cun-ent national R&D information sharing 
system w/hich provide fragmented feedback on the value of alternative^ R&D-based 
education solutions or innovations to education practitioners, researchers and 
policymakers. In doing so, she has identified many gaps and potential areas for 
improvement. 

2. Identified many challenges that will face us as the federal government changes its 
focus and treats teachers and others responsible for selecting and using the best 
R&D materials and programs as wise contributors and consumers of their own 
evaluation expertise. 

3. Generated creative options for OERI to consider in developing more systematic 
feedback systems to capture and share teacher wisdom about strengths and 



weaknesses of ihe innovations they have tried. 

Dr Campbell has unique qualifications to help OERI start its work in designing this 
comprehensive teacher feedback system. She has been a professor, product 
developer, researcher, and professional evaluator working at all levels of education in 
her distinguished career spanning more thp.n 25 years. Her specific expertise in using 
technology in education and promoting equity have also been incorporated in many 
aspects of the design specifications. 

However, due to the limited OERI funding ($2,450) for this project and the related fairly 
narrow scop(^ of work, none of us view this as the final comprehensive desif^n or even a 
total list of design options. Instead, Dr. Campbell has provided us with a valuable 
"jump start". OERI will need to build on this initial work to design this component of its 
new and more comprehensive national education dissemination system. Specifically, 
OERI will need to give more thought to including these educator feedback and sharing 
functions as it: (a) develops new internal procedures to identify and share the best and 
(b) funds old and new components of the National Education Dissemination system 
(such as ERIC and The Teacher Research Dissemination Demonstration Program). 



The following list is intended to add to the many insights from Dr. CampbelFs report, but 
it is certainly not the definitive set of issues relating to this complex, but important topic. 

D We may need to pay more attention to the complexity of what is to be 
evaluated and shared with other education consumers. 

This report focuses on innovations, but doesn't suggest how the system designers 
might think about how to handle obtaining or using the evaluation feedback on the 
different forms of these innovations. These innovations range from: 

• practices or ideas (such as cooperative learning) which are often based on 
insights from research and/or repeated experiences of practitioners to 

• transportable products or instructional materials to 

• complex multi-dimensional programs which are likely to be adapted and 
somewhat transformed by each user, rather than adopted and replicated exactly 
as the initial developer may have intended. 

It also doesn't discuss the complex "interaction" effects connected with using these 
innovations along with other innovations. Nor does it discuss the effects of using 
the innovations in different types of user contexts such as with student populations, 
teachers or school organizations that differ from the initial innovation design 
context. 

viii 
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D We need to expand Dr. Campbell's examination of the role of the decision 
makers in collecting and using evaluative information on the strengths and 
weaknesses of the multiple innovations that generally exist to address similar 
needs. 

Thus, we will need to learn more about if and how system users will: 

• use "raw" grassroots feedback on evaluations of individual resources from 
databases 

• use interpretive syntheses of the feedback on each resource 

• use consumer report types of documents where experts compare the strengths 
and weaknesses of multiple products designed to help with the same 
educational challenge. For example, in dealing with some complex topics (such 
as alternative programs to help teachers learn to treat their students fairly) 
should OERI fund the development of interpretive consumer reports to help 
potential users analyze their own needs and understand why experts used 
certain criteria to rate the program options? 

a We need additional examination of technical/procedural options such as: 

• the value and cost effectiveness of sampling specific types of users (rather than 
collecting feedback from all volunteers) or 

• collecting and analyzing group or qualitative evaluation feedback such as 
convening focus groups in person (or via electronic networking or 
teleconferences) as well as the more mechanistic quantitative information which 
can be compiled automatically using pre-prepared computer software. 

• collecting and sharing evaluative information: (a) frorfi students, parents, 
administrators and others who might become direct participants in this consumer 
information exchange system, rather than participating through specific teachers 
or other dissemination "linkers" and (b) on the impact of the innovation on those 
involved. Dr. Campbell agrees that it is important to involve students and 
student evaluators, and parents and to obtain student and parent impact 
information to include along with other teacher provided evidence to support 
their evaluations of specific innovations. In earlier drafts she also suggested 
involving students' networks such as KidsNet. 
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• integrating a computer based system to evaluate specific innovations with other 
teacher, school, state, or national level evaluation and information systems and 
requirements such as EPIE, the Goals 2000 reforms developed at the federal 
and state levels, and the state curriculum framework efforts funded by the 
Eisenhower Program. 



□ Assuming that we are unlikely to move quickly from current fragmented 
feedback and sharing activities to one national models we need to know more 
about how existing or planned entities could serve as collectors and 
distributors of the evaluative information. We also need more advice on how 
DERI can make specific changes in the dissemination and R&D programs it 
administers to build toward the more unified model suggested by Dr. 
Campbell. 

Increased insights in these areas are particularly important for ED funded 
components of the national education dissemination system. For example, v^at is 
the cunrent role of ED technical assistance centers? As technical assistance 
centers are redesigned, v^at part of this feedback system would make the best use 
of these centers' expertise and contacts? How should ED offices operate their 
funding of national demonstration programs to obtain useful comparative 
information on different approaches designed to accomplish similar goals? 



□ We need to know how ideal model feed back and sharing systems could be 
operated cost-effectively. 

Is there any evidence that volunteers and students can help reduce costs of such a 
system, particularly for the synthesis and interpretation roles? What advances in 
technology are likely to make the operations of an integrated feedback and 
consumer information system feasible? 



□ We need to know more about the implications for changing the power balance 
by having teachers play a greater role in evaluating what they use and in 
selecting the best options that their resources will allow. 

Dr. Campbell provided a good initial discussion of how the system could provide 
incentives for teachers to participate. But additional discussion is needed on how 
this new function will use teachers* valuable time, how it might make them more 
reflective and effective teachers, how it will relate tc their increased school 
management responsibilities, or how they might become more vulnerable to 
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influence by publishers or others who provide Incentives for "good" reviews of their 
products. 

We should also explore ideas on how this feedback system could help provide a 
clear function for networks or communities of learners (including researchers, 
developers, administrators, teachers, community members) with similar areas of 
expertise such as multi-cultural education or physics. 



In summary, we hope that this working paper will stimulate your creative thinking so 
that you will help OERI decide how to best use federal leverage points to obtain and 
share systematic evaluative feedback from teachers and education consumers on the 
products and programs that are such an important part of their daily lives. 



WHOSE KNOWLEDGE IS IT?: 
INVOLVING TEACHERS IN THE GENERATING AND USING OF 
INFORMATION ON EDUCATIONAL INNOVATIONS 

By 

Patricia B. Campbell, Ph.D., Campbell-Kibler Associates 



The traditional rule is that researchers only talk and teachers only listen (Campbell 
etal, 1988). 

I. INTRODUCTION 

In an effort to change that rule, the Office of Educational Research and Improvement 
(OERI) of the Education Department commissioned a report to "guide OERI in the 
development of support and incentive strategies to obtain, synthesize, and share 
evaluative information on R&D based solutions including products, programs, and 
practices." Along with describing and analyzing key types of strategies and technical 
and psychological issues, the report provides several consumer evaluation options for 
OERI to consider as it develops a more systemic national education dissemination 
system. 

The report grew out of a literature search, interviews with people involved in feedback 
systems and electronic networking systems and a "brainstorming" seminar with OERI 
staff. Information was obtained from varied sources outside of education from Amnesty 
International to Texas Instruments. All technical recommendations were reviewed for 
feasibility by an expert in databases and networking. 



II. INFORMATION SHARING: OLD MODELS AND NEW 

With the exception of a few forma! field test situations usually operated by a developer, 
there is little opportunity for educators to systematically share information with each 
other, and with developers and decision makers, abou. what R&D based solutions work 
best in different circumstances. Yet this information is key if we are going to do a better 
job in identifying and sharing the best in education. Now, for a variety of reasons 
including the push to national standards, technological advances and even the interest 
of Congress, is the time to be able to improve our answers to the question of "What is 
best?" (Klein, 1993). 

Current Models of Information Sharing In Education 

In most research and evaluation the role of the teacher is clear. Theoretically, in return 
for serving as a subject, or allowing one's students to serve as subjects, the teacher 
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receives the results of the research or development effort and can use those results to 
Improve practice and benefit students. Seen in research and development as client, 
teachers are viewed as "the passive recipients of someone else's knowledge: rather 
than as sources of knowledge themselves or active participants in their own growth and 
development" (Darling-Hammond et al, 1993, p. 4). Neither are they seen as active, 
informed consumers who make wise choices from available solutions. 

Based on a deficit approach, even in this ideal model, the feedback goes only one way. 



Researchers/Developers — > Teachers — > Students. 

Usually there is even less communication than this, Glennan (1978, quoted in Buckley 
and Schubert, 1983) found a "persistent hostility exists between the practitioner 
community and the research community which makes it difficult for communications to 
occur between them." The teacher rarely receives results, hardly ever applies any 
results that are received and even less often provides information on the application of 
the results. Students, traditionally seen as the receivers of knowledge play even less 
of a role in this process although they are ultimately the most important component of 
education. For almost three decades the Educational Product Information Exchange 
(ERIE) has been trying to change this, arguing for learner feedback on instructional 
materials. For a short time (1974-78) there was legislation in California and Florida 
which mandated this. EPIE continues to call for Learner Enhanced Assessment of 
Products and has recently launched a pilot project to gather such learner enhanced 
assessments of electronic learning resources. If the pilot succeeds, the process will be 
applied by the States Consortium for Improving Software Selection (SCISS) which is 
managed by EPIE (Komoski, 1994). 

The model is somewhat different for researchers/developers and policy makers. Ideally 
researchers/developers provide results and products to policy makers to suggest 
programs, define problems and generally improve education. Policy makers primarily 
through funding/appropriation decisions provide feedback to researchers/developers 
on what they want to be done. 



Policy Makers Re^^earchers/Developers 



Although realistically many other variables intervene (i.e. politics, researcher inability to 
communicate results to policy makers, policy makers' unwillingness to hear bad news), 
this is at least a model that acknowledges the importance of ongoing communication 
and two way feedback between researchers/developers and policy makers. Within 
both of these models the responses of the key players in the successful, and 
unsuccessful, implementation of any innovation, teachers and students, are lost. "In 
practical terms this has meant that those with the most direct influence on children's 
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learning outside the home - namely teachers - have had little if any voice or influence 
on the policies that define the limits and to some extent the very content of their 
teaching" (Fruchter and Price, 1993 p. 60). 

Yet teacJiers have a tremendous amount of useful information about the 
implementation and impact of innovations including: 

• information on the ease of use including the amount of preparation time and 
ease of access to materials as well as necessary costs and resources 

• the degree to which the innovation fits into the existing curriculum and the school 
culture 

• student respon^ie including student behavior, student remarks, student 
interest/motivation and student achievement (i.e. test scores, quality of projects). 

There is much important teacher and student generated information waiting to be 
tapped - information that can be used by teachers, administrators and policy makers in 
their decision making processes as well as by researchers and developers. And as this 
information is accessed and used, it has the potential to encourage teachers to 
generate and use additional information supporting a collaborative model of teaching 
based on notions of teachers as colleagues engaged in inquiry about practice (Darling- 
Hammond et al, 1993). 

This role of teachers as reflective generators and users of information on innovation 
and on innovation strategies is a new one for many in education, yet it is a role that is 
key to educational reform and one that is coming. Indeed thinking systematically about 
practice, learning from experience and being members of learning communities 
compose two of the five core propositions of the National Board for Professional 
Teaching Standards (no date). It is seen as a basic requirement of the second wave of 
school refomn (Myers, quoted in Darling Hammond et al, 1993) and even a teacher 
duty. 

As valuable as having teachers play this role is, it does require a change. Teachers 
traditionally are "tinkerers operating independently.. .[relying] more on personal 
practical knowledge than on interaction wit:, peers" (Cousins et al, 1993). Teachers 
often have concerns about giving or receiving assistance or even information because 
of a fear of being seen as less competent or inferring that another teacher might be 
less than competent (Cousins et al, 1993). But they need to receive information and 
others need to receive their information. 
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A New Model 



A system is needed that collects information on teacher/student responses to 
educational innovations and provides cumulative information to researchers, 
developers, policy makers and teachers. Such a system would provide information in 
formats useful to: 



• teachers and others considering adopting the innovation (teachers need to know 
what other teachers are saying. They are much more apt to believe teachers 
saying it works and it won't make you crazy - or that it will - than they are 
others). 

• policy makers (policy makers need to know if it, whatever it is, is working or not, 
at school and district levels as well as in general). 

• developers (developers need to know what needs to be refined in existing 
innovations and what new things need to be developed). 

Such as system would also let user/participants know how their information is being 
used. 



The model for this type of feedback is quite different from those previously listed. 



Original Researcher/Developer 
Other Developers (can use the 
information when they develop their 
own products) 
Policy Makers 

Other Teachers (can become 
teacher/users) 

Students (will generally use through 
their teachers) 

In this model teachers and others using an innovation, which may include policy 
makers, students and parents, provide feedback to a data compiler which may be 
electronic network based, 800 number based or paper/fax/mail based. The data 
compiler provides summaries of the data from teacher/other users of a specific 
innovation or product in different formats to four populations. Those who use 
information from such a system are provided incentives which might include priority 
access or a reduction in fees, or to provide information to such a system thus 
expanding the pool of users/providers and information. While the model may be 
national in scope, it may also be regional, state, or interest group wide. For example, 
Komoski (1994) has proposed a feedback network for the national network of 
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community computer centers under Playing to \A^in and Palanki (1993) has concluded 
that collecting information about best practices at the local level and disseminating 
them across regions was one of the primary responsibilities of state governments. 
However, a national effort that could be shared across states, regions and even interest 
groups, could decrease duplication and increase usage. 



Existing Feedback and Communication Efforts 

There have been a number of efforts to provide research and development information 
to teachers and other practitioners. These include Phi Delta Kappan's Practical 
Applications of Research and research columns in such diverse practitioner oriented 
journals as the Kappan. Electronic Learning and Today's Education . Education Week 
also regularly provides information about national research studies and educational 
innovations. Federally funded efforts such as ERIC (Educational Resources 
Information Center), the NDN (National Diffusion Network) and the NIDRR Regional 
Infomnation Exchange Program (National Institute on Disability and Rehabilitation 
Research) use a variety of ways including toll free numbers, computer bulletin boards, 
video tapes, displays at conferences, training sessions, brochures and even free 
overheads to provide information to teachers. Pilot attempts to replicate the U.S. 
Department of Agriculture's Extension model in education found the model effective in 
the three states tested but there are concerns that it may be too expensive to be 
implemented on a national level (Sashkin & Egermeier, 1993). In the model, county 
extension agents make "informational house calls," are available for technical 
assistance, and provide written and other sources of information. 

Fewer ongoing efforts have been expended to collect feedback from teachers beyond 
an innovation's developmental period. One, the Northwest Regional Educational 
Laboratory collected teacher feedback on instructional software but this was not 
continued. Perhaps the best known teacher feedback system is EPIE (Educational 
Products Information Exchange). As part of EPIE, participating teachers use products 
with students and then review them for EPIE. The information is compiled and shared 
with subscribers, who tend to be those making software purchasing decisions, and with 
some developers. This program is not currently being implemented because, once 
teachers were no longer allowed to keep the software they were evaluating, many 
districts required teachers be paid for their evaluations. 

Formal and less formal feedback and sharing systems have been organized around 
specific in and out of school programs or interests. Depending heavily on newsletters 
and meetings, networks such as the ASTC (Association of Science and Technology 
Centers) Teacher Network, Math Science Network/Expanding Your Horizons, the Tack 
Program and tho IGE Network share information across members. In one teachers' 
research network, teachers not only share information but provide support and at times 
work on research cooperatively (Chattin-McNichols and Loeffler, 1989). Other 
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examples include special interest groups (SiGs) of organizations such as the American 
Educational Research Association and organizations such as the National Coalition for 
Sex Equity in Education. A primary interest in these groups is to share information 
across members, however getting that information from members io share with others is 
often a major task. One has to read very few newsletters before coming across pleas 
for information to share. 

Teacher feedback is most apt to be collected as part of the formative evaluation and 
field testing of materials, programs and innovations. These efforts can range from half 
page questionnaires to extensive interviews and observations. Depending on the 
timing of the collection of teacher information, changes may or may not be made in the 
materials. Many completed products include teacher feedback sheets and request 
users to complete the sheets and return them, however in this evaluators' experience 
few or more often, none of these sheets are returned. Any teacher feedback that is 
collected is for the use of the developers; rarely is it shared with others, even the 
teachers who contributed their information. 

There are, of course, informal networks for sharing information including the teacher 
room conversations and both formal and informal sessions and discussions at meetings 
of local and state professional organizations such as the NEA/AFT (National Education 
Association/American Federation of Teachers) and subject area organizations such as 
the NCTM (National Council of Teachers of Mathematics), the NSTA (National Science 
Teachers Association) and the IRA (International Reading Association). 



Electronic Networks 

With the advent of computer networks, there has been a rise in attempts at electronic 
educational communication both in terms of collecting feedback and encouraging 
interaction and collaboration. Many of these are project specific such as NJ BISEC*s 
TIP (New Jersey Business, Industry, Science, Education Coalition's Teacher 
Improvement Program) network or the Harvard University's Teachers' Forum, while 
others are geographically based such as Texas' TNet and Alaska's Quill. Broader 
efforts have included Common Ground, IRIS and LabNet (Roupp, et al, 1993). Within 
more general computer networks such as COMPUSERVE and America On Line there 
are education special interest groups and bulletin boards on specific topics (i.e. math 
and science, disabilities), on educational research and on general education. Recently 
Editorial Projects in Education announced through the GTE Network the availability for 
educators of an e-mail service, a database and on-line copies of Education Week and 
Teacher Magazine . 

The success of these efforts has been mixed. For example, based on low usage, the 
NJ BISEC TIP project shut down if s electronic network. Other efforts such as Girls 
Incorporated's proposed network are not yet viable. Playing To Win, an organization 

6 



18 



of community computer centers estimates that about 10% of their participants are using 
their electronic network (Jeffers, 1993). Exploration of COMPUSERVE'S teacher to 
teacher library found between July and December one to two new entries a month, half 
of which came from 'The Society of Dead Teachers." This author received no 
response to her general query to the COMPUSERVE Education Bulletin Board asking 
teacher/participants why they used the bulletin board and why they felt other teachers 
did not. Yet the Technical Education Resources Center's LabNet has over 500 
participating teachers and evaluation studies found participating teachers using 
activities suggested by other teachers (Ruopp et al, 1993). 

Keys to success in building electronic networks for teachers are: 

• 3 shared interest 

• easy access to: 

a reliable computer network 
hardware and a phone line 
a trouble shooter 

• training in how to use the telecommunications system 

• a sense of responsibility to the project/network 

• strong leadership (Levin, Kim and Riel, reported in Means et al, 1993, Ruopp, 
1993): 

There are access issues related to who uses computer networks and the degree to 
which users are disproportionately male, white and technically oriented. Much has 
been written about computer inequities in terms of class, race and sex (i.e. Sanders, 
1986; Campbell, 1985). This may be an issue in electronic networks as well. One 
article summarizing the highlights of an electronic bulletin board discussion of 
educational policy quoted 20 respondents, 17 of whom were male (Glass, 1993). 

There is a great deal of evidence of increased interest in educator use of electronic 
networks and increased teacher access to these networks. Examples include: 

President Clinton's 1994 State of the Union pledge to "connect every classroom into 
the information superhighway by the year 2000." 

Efforts of Individual states including Florida, Texas, Pennsylvania, Virginia and 
Pennsylvania to establish networks that allow teachers to communicate 
electronically (West, 1994a). 



Two companies. Bell Atlantic and Telecommunications, plan to link all the schools in 



their service area (26,000- 25% of all schools in the country) to data networks 
(West, 1994b). 

While K-12 educators remain a small segment of the user group, there are many efforts 
under way at the federal, state and local levels to increase their access to and 
familiarity with electronic networks (Education Week, 1994). These include: 

r 

Increasing teacher access to electronic networks is now a priority under the Fund 
for Innovation in Education, US Department of Education (Education Week, 1993). 

Allowing Eisenhower mathematics and science program grantees (U.S. Department 
of Education) to use up to 20% of their funds to obtain equipment and 
telecommunication services {Education Week, 1993). 

The development, broadcast and dissemination of a video tape from NASA for 
teachers on how to use INTERNET (Education Week, 1994). 

The development of a grants program from The US West Foundation and the 
Annenberg/Corporation for Public Broadcasting Math and Science Project to "train 
elementary and secondary teachers in how to use INTERNET and other major 
computer networks" (Education Week, 1993). 

The funding of five projects to bring rural teachers into the INTERNET networks 
(Education Week, 1993). 

The tremendous growth of computer networks such as COMPUSERVE and Prodigy 
which are each reporting over 50,000 new subscribers a month. Even the 1993- 
1994 football playoffs were tied into Prodigy where viewers were invited to log on 
during half time. 

At the same time there has been an increase in student oriented electronic networks. 
For example, TERC (Technical Education Resources Center) and National 
Geographic's KidsNet have been successful in providing students with an electronic 
network in which students systematically collect and report scientific data which is 
aggregated across student groups and summaries provided to participating students 
and scientists. Such a pool of data helps students look at their site data in context 
while also providing scientists with much more geographically representative data. At 
this point approximately 10,000 schools, primarily in the United States, are using it 
(Barstow, 1993). This type of student feedback and central compiling and synthesis of 
student collected information will be increased dramatically and internationally with the 
GLOBE program under the auspices of Vice-President Gore. 

Public or private, in school or out of school, electronic networks are with us. While 
students may be ahead of teachers (in one district the students actually wired the 
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network and taught the teachers how to use it (Fortune, 1993)) and administrators may 
currently have more access than teachers (West, 1994b), the country appears to be 
committed to putting schools and teachers on the information superhighway. 

This commitment has great implications for the ways that information is collected and 
disseminated. Not only does it mean that teachers, and others, will have the potential 
to access huge amounts of information, it can mean that these same teachers can more 
easily provide information to others. This information can be summarized and shared 
in different formats with researchers and developers, policy makers and practitioners. 
This may be done automatically, under the direction of a computer program designed to 
summarize the data, or under ths directJon of educators/researchers. While the 
automatic summaries would be faster and less expensive, the "human summaries" 
could include interpretations and analysis. 

The growth of electronic networks has the potential to make a model of two way 
communication and information sharing possible. The technical challenges of setting 
up such a system are not great. But, as the following section describes, the societal 
and psychological challenges are. 

III. CHALLENGES TO FACE 

Changing the Focus: Teacher. Student and Policy Maker as Customer 

In education as in government we have tended not to see teachers and students or 
even policy makers as customers. Yet they are. If teachers don't "buy" or use our 
results or innovations or if teachers try a product or innovation and are dissatisfied and 
have no recourse, then we have not been successful. As with business we want 
customers to try our product and to have it work well so they will continue to use it and 
even tell others. Yet there are always "kinks" ~ things that are unclear, problems or 
things that are just wrong. When this happens in other areas of life we often end up 
contacting the manufacturer's Customer Service Department. 

Industry, particularly the computer industry, gets much of it's feedback on problems and 
needs from people who are confused or in trouble and call customer service. While I 
am not going to call to tell Word Perfect what I think of their product, I will call if I can't 
figure out how to get the copyright sign or if my screen locks when I push a combination 
of keys. And while I am getting my questions answered I'm willing to talk more 
generally about Word Perfect as well. I get something I need. Word Perfect gets 
something it needs and during the exchange, my information is entered into a database 
for later summary and analysis. 

There are two ways that I, the customer, get to access customer service. For some 
products all I have to do is buy it and I get the customer service numbers. Others 
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request 1 fill out the warrantee card first. By doing this I get access to free customer 
service for a specified period of time and the company gets additional information from 
me. In education we don't have customer service departments, but we need to. And as 
teachers and others call, for example, the National Council of Teachers of Mathematics 
with questions about the implementation of the standards or call the Technical 
Resources Information Center with questions about the implementation of KidsNet, they 
can be asked questions about their response to the innovation and areas in which 
improvement could be made. This information could be summarized and used by the 
developers for revision purposes and by other consumers or potential consumers. 

In From Red Tape to Results . Vice President Gore said that the government will 
''require that all federal agencies put customers first by regularly asking them how they 
view government services, what problems they encounter and how they would like 
services improved'' (1993, p. 44). This is an unprecedented step in the collection of 
user feedback - yet, if the user doesn't get something out of it - a reward, immediate 
information that the user needs, or specific indications that the information the user 
provides is being listened to and acted upon, then the quality and quantity of data 
needed for such a system to work will not be collected. 



Answering the Question: What's In It For Me? 

All of us - bureaucrat or business owner, cabinet secretary or office clerk - 
respond to incentives. We do more of wtiat brings us rewards end recognitions, 
less of wtiat brings us criticisms (Gore, 1 993. p. 43). 

Vice President Gore's quote may seem obvious but it is not. In education we tend to 
believe that teachers don't need incentives, that teachers will "do it for the sake of the 
children" independent of incentives or disincentives. Xet teachers, too, do what brings 
rewards and recognition. Thus, for this system to work it must address "what's in it for 
me?" 

There are a variety of incentives that can be used to encourage participation including: 

• "paying" for access to information by providing information 

• receiving free trial materials and technical assistance for pilot or "beta" testing 

• receiving "coupons" for lower cost or free related educational materials 

• receiving recognition as a valued professional innovation reviewer 

• receiving feedback about how the information is being used by others and its 
impact 
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• knowing that your experiences and evaluations as an Innovation reviewer are 
being taken seriously and used by others 

• being paid. 

These incentives have all been used with varying degrees of success. They need to be 
systematically tested and compared for relative effectiveness. 

For too many years teachers have viewed research and evaluation as a "black hole" 
where their efforts and those of their students' (and many, many reams of paper) have 
gone somewhere for some purpose but they don't know what. It Is very difficult to keep 
doing something and doing it well, if you receive no feedback about the results of your 
action. 

This was an issue facing Amnesty International whose members regularly write letters 
to foreign governments asking for the release of prisoners and investigations into 
deaths or ill treatment. Letter writers rarely receive any response from the 
governments and when they do, it tends not to be positive. Amnesty keeps letter 
writers motivated by periodically providing information about the release of prisoners, 
thanks to the letter writing campaigns. In a very concrete way this tells people that 
what they are doing is important and has an impact. 

Providing teachers/users with specific information on how the feedback is being used, 
including occasional stories from developers and implementers on how things changed 
because of the feedback, can both motivate participants to continue and perhaps 
encourage others to join in. 

Being listened to and being taken seriously can be an incentive as well. Unfortunately 
being seriously listened to by educators is not that common an experience for teachers. 
Indeed no teachers wer-^ invited to participate in the initial National Education Summit. 
Not surprisingly teachers participating in research studies reported they liked to talk to 
researchers who actively listened to their concerns and that they were moved when 
their expertise and concerns were sought out by the researchers (Beyerbach, 1989). 
To have evidence that they are being listed to can be a powerful motivator. 

Being listened to by follow teachers is one thing, beirig listened to by policy makers and 
researchers and developers is another. For any system to be more than a formal 
teacher exchange, there must be "buy in" from policy makers and researchers and 
developers to access and where appropriate use the information, and to feed back to 
the system how they use or are planning to use that which they accessed. 

It is easier to get teachers to do things, if there is a direct relevance to their job; they 
also do things that are required. It is possible to require teachers to try a feedback 
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system but unless it is useful to them - that is direct, concrete and classroom specific - 
teachers will not continue. 



Possible Ways of Selecting Innovations to Include : 

There are a number of ways that innovations could be identified to be included in a 
feedback system. These could include: 

• nomination by developer 

• nomination by a team of practitioners and researcher/developers 

• nomination by at least 10 practitioners 

• nomination by self and others combined with further exploration by designated 
experts 

• innovations seen by the Education Department or other sponsor as widely used 

• self-nominations or applications to review panels such the Department of 
Education's Program Effectiveness Panel which identifies effective programs for 
dissemination by the National Diffusion Network and types of review groups. 

• selection of innovations that are already receiving federal support for 
development or demonstration purposes. 

• special national search procedures to find the best products or programs 
intended to fulfill specific purposes such as to teach reading. 

More thought needs to be given to the fairness, utility and cost implications of including 
innovations in these feedback systems. For example, if developers and commercial 
publishers are encouraged to put their own products in such a system, will they also be 
asked to bear some of the costs? If so, would exceptions be made to encourage 
teacher developers and others with less access to evaluation/marketing funds? 

Since it is doubtful whether initially users would input their own or other adaptations, 
the focus on more widely used or promising innovations would seem more logical. It is 
possible that feedback systems could be tied into a database of innovations where 
there is electronic access to available innovations, curriculum materials and products. 
While this combination and expansion of a variety of existing, less formal efforts would 
be a somewhat different project, it would facilitate the spread of information on 
innovations as well as the collection of feedback on their use. 
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Possible Types of Information to Collect : 

Any system established should, most likely, focus on the collection of evaluative 
Information; primarily teacher and administrator perceptions of what worked, what didn't 
and what could be made better. There could be opportunities for users to input impact 
data. This would be at best a supplement to research results, however the 
complexities of summarizing and assessing the quality of this data suggest that this not 
be part of any initial efforts. 

There are a variety of types of information that could be collected from users. These 
could include structured and open-ended responses such as: 

• Innovation name 

Unit of innovation (i.e. classroom, department, school, district) 
Subject area(if appropriate) 
Grade level (if appropriate) 
Track (if appropriate) 

• Demographics for the unit of the innovation (i.e. class, school, district) 

Size 

Geographic location 
Rural, suburban, urban 
Ethnic and gender breakdown of population 
^ SES of population (free lunch or some-other measure) 

• Ratings of innovation 

Overall Rating 
Ratings of: 

Student Instructional Materials 

Teacher Instructional Materials 

Content: 

Accuracy 

Degree fits in existing curriculum 
Student affective response 
Student cognitive response 
Ease of access to necessary materials 
Amount of prep time 
Ratings compared to appropriate competitors 

• Training: 

training needed 

quality of training provided 

comprehensiveness of training provided 
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• Amount of administrative support needed 

• Open-ended or Checklist Responses 

What are the strengths of the innovation? 

What are the major problems encountered implementing the innovation? 
Which were solved? 

What were the major effects on students? Were there differential effects for 
different groups of students such as girls and boys, higher and lower achieving 
students? 

• Open-ended 

What advice would you give to others trying this innovation? 
How were major problems solved? 

Systems could also be designed such that innovation users could indicate if they would 
be interested in being contacted by other educators interested in adopting/adapting the 
innovation. As part of the set up of any system, a number of teachers, possibly through 
NDN, should provide information on their response to different innovations. Before any 
system "goes public" it should have a base of information and feedback so initial 
inquirers would receive useful information. 



Possible Types of Information to Be Fed Back 

Regardless of the system established, no information on an innovation should be 
provided until a selected number of teachers and learners have provided input. 
Conclusions should not be drawn on the responses of one or two people. To collect an 
initial pool of information, teachers who are using specific innovations might be 
recruited by developers, professional organizations or existing computer networks to 
provide feedback for a small fee. 

It would need to be decided as to whether all information provided by teachers, and 
others, would be included in the summaries or if there would be a professional team 
who would sift the data, v/eed out data that was not appropriate and, if necessary, 
recruit additional teachers to review the innovations. 

To some degree the type of information provided would need to be targeted by the 
inquirer. For example a state education department official might be interested in the 
aggregated response to GESA (Gender/Ethnic Expectations Student Achievement) 
while a principal might only be interested in implomentation data from urban elementary 
schools. If there were not enough data from urban elementary schools to be given 
alone, then the principal would receive the overall summary. 

Also some inquirers might only want summaries of teacher ratings, while others, 
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particularly teachers and developers, may be more Interested in the open-ended 
responses and the opportunity to speak with someone who has implemented the 
innovation. It is important to build .this kind of flexibility into whatever systems are 
developed. 



A Possible Model for An Innovation Feedback System 

The following is a description of one possible model to obtain feedback on promising 
and exemplary education innovations. In this model inquirers could request a list of 
innovations covered by the system by such variables as unit of innovation (i.e. 
classroom, department, school, district), subject area and grade level or enter the 
innovation. The information they would receive could include: 

• a 2-3 paragraph summary of the innovation including impact evaluation results 
and information on where to call/write for information on implementing the 
innovation including sources or training and materials. (This would have been 
put in the system by staff or developers.) 

• the number of teachers/other users who have provided information on the 
innovation and the types of students/schools over which the innovation has been 
used. 

• the number of people who have accessed the feedback and their reported use of 
the information received. 

• "objective" information summaries. This information may be reported in total, 
disaggregated by such variables as geographic location, population SES, or for 
a particular location, or SES group if the cell sizes are large enough to make this 
appropriate. The information summaries would include: 

• a summary of user ratings in each category - including means and frequency 
counts 

• a summary of user check offs 

• graphic representation of the data. 

• open-ended summaries: 

Inquirers can choose to receive responses to any or all of the open ended responses. 
Ideally they would receive a one to two page summary that focuses on the reoccurring 
issues and themes generated in the open-ended responses along with sample quotes. 
However it is not clear how feasible this would be; the person hours necessary to do 
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this analysis and keep it updated would be very high. Other possibilities include: 

• the inquirer being able to do a key word search and receive unedited open- 
ended responses 

• the inquirer receiving a random sample of open-ended responses. 

If they desire, inquirers could also be given the names and telephone numbers of two 
or three innovation users who implemented the innovation with populations similar to 
those of the inquirer and who would be willing to discuss the innovation. 

Other models might include subsets of the above model or include more access to 
"raw" teacher and administrator data or could focus on expert interpretations of the 
data. Certainly whatever information was provided would need to be updated 
periodically. If the summarizing system was programmed into the system, this would 
not be a major issue. If however experts were needed to review new data and possibly 
revise the infomnation provided to inquirers, it would be necessary to schedule this 
either based on time (i.e. a new update every six months) or new data (i.e. a new 
update each time 25% more data is received). 

Regardless of the types of information provided in feedback, the inquirer could be 
asked to input, perhaps using a checklist with an option for an open-ended answer, 
how they are planning to use the infomnation. To increase the amount of data available 
to the feedback system, inquirers could also be asked if they have information on any 
innovations they have tried and if they would be willing to input their information. They 
could then be provided the opportunity to input their information. 

Getting the Word Out: Selling The Concept of A Feedback System 

Key to the success of any feedback system is letting people know it is available. 
Certainly to do this the traditional modes of dissemination should be tapped - ads and 
articles in professional publications (i.e. Education Week , Today's Teacher Science 
Teacher . Young Child ) and regional and state newsletters. Feedback systems should 
be demonstrated at national and state conventions of the AFT, NEA, NSTA, NCTM, 
IRA and NOTE (National Council of Teachers of Engli^^h), where teachers have 
opportunities to enter and receive information. This could also serve as part of pilot 
efforts to test various input, output and incentive strategies. 

Word about any feedback system could also be distributed through existing networks, 
both commercial networks such as COMPUSERVE, Prodigy and On Line America as 
well as professional networks including ERIC, INTERNET, BITNET and the National 
Science Foundation's STIS (Science and Technology Information System). This might 
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include a short tutorial on how to get into the systems from the specific network (if that 
is possible) as well as how to use them. 

These efforts would reach most developers and researchers and many teachers. 
However to reach most teachers it will be necessary to tap into the general media; 
through magazine and newspaper articles and even public service announcements that 
give people information about feedback systems and ways to access them including, if 
appropriate, the 900 telephone number. And never underestimate the power of a 
famous spokesperson for a system. 

The process of getting the word out and getting teachers and other users involved 
should depend heavily on Baker's (1987) tactics of innovation. Baker reminds us that 
by using words and concepts that people are already familiar with, publicity efforts 
should stress the advantages of any system used, it's simplicity to the user and it's 
reversibility (once in the system you can get out of it). 

IV. TECHNICAL ISSUES: CAN A TEACHER-FRIENDLY, ELECTRONIC NETWORK 
BE DEVELOPED? 

Issues of cost and system/user/inquirer interaction mean that ultijr.ately any 
comprehensive system must involve an electronic network. However it is clear that 
beginning only with an electronic network will leave many teachers "out of the loop." 
Initially any system should include multiple modes of input and output to "hook teachers 
in." Modes of input could include: 

• An 800 Telephone Number: 

Alternative I: Users with push button phones would speak in the name of the 
innovation and their name, school, address and so on. They would enter 
innovation rating and check off data on an innovation. The current status of 
voice recognition software would mean that the taped information would have to 
be listened to and entered into the database by hand. Users would then be 
given numeric and alphanumeric code based on their name/zip code they would 
use when providing additional information or when accessing information. 

Alternative II: Users would speak directly to an operator who would enter the 
information into the data base during the conversation. 

• Marl/Fax: 

Information forms would be widely distributed to teachers and others through 
professional organizations, unions, newsletters and other sources. These forms 
would be mailed or faxed to a central location. There are a variety of problems 
with this mode including all information must either be scanned in and checked 
or entered by hand. In addition there is no potential for interaction between the 
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users and the system. Too. it is most difficult to motivate users to use this 
system. 

• Electronic Network: 

This is of course the best option, but as indicated earlier, at least initially it 
should not be the only option for any system that wants to reach a representative 
group of educators. The network could be accessible through existing computer 
networks including INTERNET, AMERICA ON LINE, COMPUSERVE and 
BITNET. To encourage use of the electronic network, users of other options 
could be provided with information on how to access the network. Users who 
used the network to input information could be given extra incentives including 
perhaps greater or cheaper access to information, greater reduction of rates on 
materials and so on. 

Output Options Could Include: 

• A 900 Telephone Number 

Inquirers would give an operator or a tape their name, school, address and so 
on and the innovation about which they would like information. The information 
would then be mailed or faxed to them. Their telephone number would be billed 
for the cost. If the inquirers were also users, they would give their code and 
either would not be charged or would be charged at a lower rate. 

• Fax/Mail 

Inquirers would fax or mail requests for information to a central location where 
their requests would be filled. If there is a fee, inquirers would have to include a 
check or credit card authorization with their request. If the inquirers were also 
users, they would give their code and either would not be charged or would be 
charged at a lower rate. 

• FaxTTelephone 

Based on a system already in use by computer companies, inquirers would 
telephone in, on their fax line, using a push button phone to punch in their fax 
number and the innovation(s) about which they want information. Pushing the 
receive button on their machine, the innovation summaries would then be 
automatically faxed back to them. 

• Electronic Network 

As was the case with the input options, this is the best option, but again at least 
initially it should not be the only option for any system that wants to reach a 
representative group of educators. The network would allow for immediate 
response and interaction as well as being significantly cheaper. To encourage 
use of the electronic network, inquirers who used other options could be 
provided with information on how to access the network. Inquirers who have 
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provided information to a system could be given cheaper rates. 



At some point, regardless of the system developed, for both cost and quality reasons, 
designers should consider phasing out telephone, telephone/fax and fax/mail 
input/output options and having all interaction's conducted through the electronic 
network. To use a network it would be necessaiy to have: 

telephone access 

a computer with a modem and communication software 

either direct access to the network or indirect access through another computer 

network. 

Most people need a person with technical experience to set up communicatiot js 
software for them. The logical place for that to happen is in the school or the district. 
Almost all schools now have computers and most of them have modems. However, 
during the school day is when teachers have the least amount of time and least access 
to telephone lines. For teachers to use the network through the school - it would have 
to be before school, after school or during prep periods. 

Teachers might use their home systems. However while many teachers have home 
computers, fewer have modems and even if communication software were given to 
teachers, many would be unwilling or unable to set up the software. Once the software 
is setup for a computer, it is very easy to get on a network. For example with 
Magicsoft, once it is installed with the log-on numbers and passwords put in the 
software, to enter COMPUSERVE takes one click of the mouse and two presses of 
enter. Everything else including dialing is automatic. However it takes some skill to set 
up the program to do this. One possibility is for the Department of Education to work 
with a software company or a company that manufactures modems, such as Hayes, to 
develop easy to install communications software for relatively simple computer systems 
that do riot have, for example, on-line fax software. The Department could subsidize 
the development of the software and recommend it's use or purchase. For some 
schools such as Chapter I schools, the Department could distribute a certain number of 
copies of software. 

While this paper is focusing on systems to collect and disseminate teacher information 
on innovations, such a system could and should tie into other systems under 
consideration. Hooking a teacher feedback system into a database of existing 
innovations would help in the dissemination of information about innovations. Working 
with software and hardware companies to develop teacher friendly telecommunications 
software is an important step to hooking teachers and schools into the information 
superhighway Too, any feedback syr*em that is developed could be a part of a larger 
system that included research and impact evaluation data on innovations. As a variety 
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of systems are contemplated, it is important to assume that in some ways these 
systems will be integrated and to set up their designs so that this is possible. 

What is being proposed is not a "stand alone system" dedicated only to collecting 
feedback on innovations; rather this should be seen as one aspect of a broader based 
effort to connect schools, policy makers, researchers and developers as education is 
restructured. 



V. RECOMMENDATIONS 

There are a variety of low and no cost steps that can be implemented now while 
discussions are begun about the feasibility of a national system. While there is much 
to be gained from a national system, one does not have to be implemented to increase 
the collection and use of teacher feedback. 

An important early step is to include the need for teacher feedback as a dissemination 
standard for the National Science and Technology Council and its Committee on 
Education and Training which coordinates federal efforts in these important areas. This 
will begin to increase awareness of the need for and the value of teacher feedback. At 
the same time a panel of teachers, developers, dissemination specialists and computer 
network experts should be convened to work on the development and implementation 
of an entire feedback system. 

Simultaneously a number of pilot teacher feedback efforts can be relatively easily 
established. These can include: 

A. Making teacher-based feedback loops part of existing programs. 

National Dissemination System: The collection of teacher feedback can be 
incorporated as part of National Diffusion Network and related federally funded 
dissemination efforts. As part of their NDN funded promising or exemplary program, 
its directors could collect a common core of information about teacher and student 
responses to share with other potential users. They may also collect additional 
information for their own use in revising or adapting the program. The raw data, 
preferably in machine readable form, is provided to the Department or to program 
developers or others for summary, analysis and broader dissemination- perhaps 
through the ERIC system. 

NIDRR Regional Information Exchange Program: As part of their evaluation effort 
which documents activities and adoptions and the involvement of family members, 
consumers and members of cultural minority groups, participant feedback can be 
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collected, summarized and provided to interested consumers as part of tlie 
Exchange Program's dissemination efforts. 

Labs and Centers: Labs and Centers can be required to collect feedback from 
teachers and others on their own materials and those of other programs and 
Innovations and to provide summaries of the feedback in their newsletters and other 
dissemination efforts. In addition Labs and Centers can serve as "feedback 
centers" collecting and disseminating teacher feedback about other organizations' 
products and programs as well. In cooperation with the National Diffusion Network, 
the Laboratory Network Program has already done some of this in connection with 
their recent publications: Promising Practices in Mathematics and Science 
Education (1994) and Mathematics, Science and Technology Programs that Work 
(1994). 

ERIC: ERIC can be used to disseminate feedback on innovations collected by other 
sources. Currently, ERIC commissions and disseminates papers on a variety of 
topics. They have commissioned and disseminated papers and consumer reports 
such as Secondary Special Education: A Guide to Promising Public School 
Programs (Warger & Weiner, 1987) and could do more to collect and synthesize 
teacher responses to different innovations. 

B. Set up a mini feedback system. 

There are existing electronic education networks, including COMPUSERVE'S 
Education Forum (SIG). In much the same way that the American Educational 
Research Association set up an educational research bulletin board through 
COMPUSERVE, the Department could set up a mini feedback system through one 
of the existing electronic networks (i.e COMPUSERVE, INTERNET, AMERICA ON 
LINE) where teachers would rate specific innovations on 2-3 variables and give 
some open-ended responses that could be accessed through key word searches. 

C. Develop and disseminate summaries of teacher feedback currently being done 
through Department projects. 

Formative evaluation results from existing projects can be collected and one-two 
page summaries can be written aimed at the interests and information needs of 
specific populations. These can be disseminated through ERIC and/or through 
existing network bulletin boards. 

D. Develop a teacher feedback panel. 

Drawing on expertise and good will of teachers already involved with the 
Department as consultants, grantees and award winners, specific teachers can be 
asked to provide and feedback information to the Department on innovations they 
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have tried. These same teachers can be paid a small honorarium to collect similar 
information from other teachers. Again one to two page summaries can be written, 
aimed at the interests and information needs of specific populations which can be 
disseminated through ERIC and/or through existing network bulletin boards. 

E. Modify the review criteria for proposals in grants announcements to include the 
collection of feedback from teachers. 

Currently most Education Department programs require program evaluation. As 
part of that evaluation, the collection of teacher responses to the 
program/innovation could be required including even a common core of questions 
that could be answered to help educators compare programs/innovations. This 
information would be included in project reports with a summary provided with 
program/innovation dissemination aborts. 



In conclusion, big system or small, electronic or not, state wide or national, all 
innovations or a selected subset, we need more information on what happens when 
innovations are implemented. We need it to make better policy decisions, we need it to 
develop better innovations and we need it to make better decisions on what innovations 
to use. There are things we can do now to better collect and share teacher information. 
We need to begin. 
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APPENDIX: SOME KEY CONTACTS 

Teacher/User Evaluation of Materials and Software 
Kenneth Komoski 

Educational Products Information Exchange 
1 03-3 W. Montauk Highway 
Hampton Bays, NY 11946 
516 283-7590 



Teacher Networks 
Andrea Anderson 

Association of Science and Technology Centers 
1025 Vermont Ave., NW, Suite 500 
Washington, DC 20005 
202 783-7200 



Teacher/Educator Electronic Networks 

Laura Jeffers 

521 West 111th St. 

New York, NY 10025 

212 666-2761 

Richard Ruopp 

Technical Education Resources Exchange 
2067 Massachusetts Ave. 
Cambridge, MA 02140 
617 547-0430 

Margaret Honey 

Center for Children and Technology 

Education Development Center 

96 Morton St. 

New York, NY 10014 

212 807-4200 

Nancy Stefanik 
P.O. Box 321 
Newfane, VT 05345 
(802) 365-7292 
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Teachers as Members of Communities of Learning 

Linda Darling-Hammond 

Ann Liebermann 

Teachers College NCREST 

Columbia University 

New York, NY 

212 678-3763 



Dissemination of Program Information 

Max McConkey 

Executive Director 

National Dissemination Association 

4732 North Oracle Road, Suite 217 

Tucson, AZ 85705 

602 888-2838 

Susan Klein 
OERI 

555 New Jersey Ave., N.W. 
Washington, DC 20208-5645 
202 219-2038 



Technical Issues On Databases and Networking : 

Tom R. Kibler 

Symmetrix 

One Cranberry Hill 

Lexington, MA 02173 

617 862-3200 
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